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Final Report 
Section 6.1: “ 


for re-evaluating 
replaced, no pro 


(October 2003) 

Board notes that although there is a process for 
analyses when the system is designed and a process 
them when a design is changed or the component is 


»» 


- Section 7.1: “Signals of potential danger, anomalies, and critical 
information should, in principle, surface in the hazard identification 
process and be tracked with risk assessments supported by engineering 
analyses. ” 


- Finding 7.4-5: “Risk information and data from hazard analyses are not 
communicated effectively to the risk assessment and mission assurance 
processes. The Board could not find adequate application of a process, 
database, or metric analysis tool that took an integrated, systemic view of 
the entire Space Shuttle system.” 


2006 ASAP Annual Report in regards to Safety Management 
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Ordinarily, accidents are prevented by a 
barriers (human and hardware system features 
prevent accidents) 



Some holes due 
to active failures 



Other holes due to 
latent conditions 
{resident "pathogens") 


Successive layers of defences, barriers and safeguards 

• Complete pathway through cheese represents acciden 

• Precursor conceived as partial pathway through the 
holes in Swiss cheese 

• Precursor analysis then corresponds to learning about 
existence and size of holes 


Definition of a “precursor” 

- An indication of a problem with the potential to recur 
with more severe consequences 


Key Attributes: 

- Observation indicates some failure mechanism 

- Same mechanism could occur again 

- The consequences could be more severe than what 
has been experienced 


Columbia 


Davis-Besse 

NPP 


Anomalies 

Debris impacts on 
thermal protection 
system 

Frequent || 
Containment || 
Air Filter || 
Replacements If 


Other Observed 
Anomalies 


(Potential) Failures 


Severe Burn-through, ET 
containment compromised 
Loss of Shuttle 


Severe RCC 
Impact, Loss < 
Shuttle on re-ei 


Significant 

Vessel 

Head 

Erosion 
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A near-miss because of chance or an opportune 
mitigation 


Faults that can become failure conditions without 
correction 


• Unexpected trend in test, operation, or maintenance 

• Unexpected effects from aging of equipment 

• Common causes of faults or deteriorations 
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• Establishes a systematic process for evaluation of flight and 
test anomalies 

- Risk-based evaluation of failure mechanisms 

- Triggered by actual flight/test experience 

- Emphasizes ‘imagination’ through generalization 

• Provides insight into safety performance 

- Identifies safety-related system vulnerabilities 

- Indicates trends in safety performance 

• Makes safety analysis more experience-based 

- Triggers review/modification of safety models based on 
analysis findings 

• Completeness of represented failure modes 

• Failure probabilities and influencing factors 
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•Feedback 



Safety and 
Performance Model 
•PRA 

•Reliability Model 
•Trending Model 
•Accident Precursor 
Model 
•Others 


•APA Allows 

Improve the 
Fidelity & 
Accuracy of 
ddr Models 


Good FH^Or Not? 


Collection and Analysis 
of Operational Data 
(e.g. failure, faults) 
Supports Development 
t>f Models 


Results of the Model 

•Predicted System 

Performance 

•Trends and Temporal 

Behavior 

•Risk Importance 

•Risk/Failure 

Significance 

Determination 

•Others 




Our Systems 

•Shuttle 

•ISS 

•Robotic 

Missions 

•Ground 

Facilities 


Operational Off" 
Nominal Data 

•In-flight Anomalies 

•Non-conforman^: 

cases 

•Failures 

•Faults 

•Others 


•Updating The 
Model based on 
Experience 


Management Decisions 
and Actions 


Risk 
information Input 






Historically, precursor analysis has been focused on 
failures, e.g., at Nuclear Regulatory Commission 


NASA process extends focus to anomalies 

- NASA’s databases contain mostly anomalies 
(a defect, fault, or other deviation) 

- NASA has a stronger incentive to prevent any failure 
due to fewer barriers in its space systems 


Operational definition of precursors: 

Anomalies that upon evaluation are determined to 
indicate a failure mechanism that may pose a significan 
degree of risk 





Screening and Dispositioning 


Screen -y«, — ►D ispositioning- 

Grade the potential 
impacts to safety — i 


Generalization 


o Apply the mechanism to 
different circumstances 


No Further Action 


Modeling 

Quantify the 
impacts 


Observation 
J& Trending 



Findings 

Complete results, 
reconciling the model 
with reality 


11 








Mechanism 
active within 
subsystem or 
component 



Potential for 
Failure 
Conditions 
of Concern 



Potential for 
Severe 

Consequences 


Other failure times 
Other systems • 

Other locations on the affected system • 
Larger fault magnitude • 

Anomaly failure mechanism • 




Problem 

Potential 

Index 
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* Anomalies without obvious risk- or reliability implication 
are removed from consideration using rules-of-thumb 


Failure mechanisms of 
screened-in anomalies 
are determined and 
generalized 

Dispositioning is based on 

- Generalized problem 
potential 

- Evidence caliber 





Decreasing Evidence Caliber 
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Parametric Probabilistic Modeling (PPM) 



Parametric Probabilistic Modeling & Risk Significance 
Assessment 


- Quantify the risk potential for anomalies and GPFs dispositioned 
for Risk Modeling 



Provides a rigorous assessment of the quantified risk 
significance of the failure mechanism acting within the 
system while also highlighting parametric uncertainties 
of the accident sequences that should be further 
investigated PPM Resu|ts 


Anomaly Risk 
Importance 
& 

Parameter 
Vulnerability 
Analysis 




•Parametric 

-Model 


Event Sequence ‘i 
Model 


.TWitUf.. < rWetN . 


Scenario Development for Events 
Dispositioned for Risk Modeling 


Anomaly 

Contribution 

Nominal 

Contribution 
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Rule-Based Screen Out 


Observation & Trending 
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• Accident Precursor Analysis (APA) has been used 
by other govt agencies with positive results (e.g., 
NRC) 

• Intended to be applied outside the normal problem 
resolution cycle 

• Development of NASA APA methodology will 
continue in 2010 




